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ABSTRACT 

Analysis based reconstruction has recently been introduced 
as an alternative to the well-known synthesis sparsity model 
used in a variety of signal processing areas. In this paper 
we convert the analysis exact-sparse reconstruction problem 
to an equivalent synthesis recovery problem with a set of 
additional constraints. We are therefore able to use existing 
synthesis-based algorithms for analysis-based exact-sparse 
recovery. We call this the Analysis-By-Synthesis (ABS) ap- 
proach. We evaluate our proposed approach by comparing it 
against the recent Greedy Analysis Pursuit (GAP) analysis- 
based recovery algorithm. The results show that our approach 
is a viable option for analysis-based reconstruction, while at 
the same time allowing many algorithms that have been de- 
veloped for synthesis reconstruction to be directly appUed for 
analysis reconstruction as well. 

Index Terms — Analysis sparsity, synthesis sparsity, 
sparse reconstruction, analysis by synthesis 

1. INTRODUCTION 

In recent years, sparse representation of signals has been an 
active research domain in signal processing. Until recently 
the usual sparsity model considered was a generative model, 
known as synthesis sparsity: a signal a; G M'' is sparse if it 
can be expressed as a weighted sum of a few signals (called 
atoms) from a known dictionary D € M''^^ 

X = Djs, with II75II0 = k (1) 

where ||-||o represents the io pseudo-norm, defined as the 
number of non-zero coefficients of a vector. The decomposi- 
tion vector 75 is thus required to have k non-zero elements. 

Lately, a different sparsity model known as analysis spar- 
sity has been proposed [IJ, asserting that the signal x produces 
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a sparse output 

7^ = 17x, with||7^||o-iV-/ (2) 

when analyzed with an operator e R^^'^, where I is the 
number of zero coefficients of 7^1 (see Section |2] for more 
details). 

Both of these models can be successfully used as regu- 
larizing terms for ill-posed inverse problems. In this paper 
we focus on reconstructing a signal x that is observed only 
through a set of m < d linear measurements, arranged as the 
rows of an acquisition matrix M £ W"^'^, possibly affected 
by noise e 

y^Mx + e. (3) 

This is known as the compressed sensing problem, which has 
been extensively studied |l2]|3l and used in practice in various 
applications (e.g. |H1|5|). It is now well known [2 | that a suf- 
ficiently sparse signal x can be efficiently recovered from the 
measurements y by solving the synthesis-based optimization 
problem 

X = Dargmin II75II0 with \\y - 1/1375112 < e (4) 

7s 

where e is the estimated noise energy of the measurements. 
Interestingly, it has also been shown [6| that a sufficiently 
sparse 7^ in (|2]) also allows accurate recovery of the signal 
X by solving 

X = arg mm WnxWo with \\y - Mx\\l < e. (5) 

X 

Problem (|4]) is NP-complete |7|, and we hypothesize that (|5]l 
may be similarly difficult. However, under stricter conditions, 
the io norm in both equations can be replaced with the ii 
norm, leading to convex optimization problems that are much 
easier to solve Q . 

In this paper we pursue an approach to solving the 
analysis-based reconstruction problem (|5]l by rewriting it 
as an equivalent synthesis reconstruction problem. The paper 
is structured as follows. Section |2] contains a closer look at 
the analysis model and its details. In Section|3]we propose the 
Analysis-By-Synthesis (ABS) scheme for exact-sparse analy- 
sis recovery. In Section]?] we compare our approach with the 



results obtained with the Greedy Analysis Pursuit algorithm 
191 designed to solve (|5]) directly. Section [5] contains further 
considerations regarding our scheme. Finally, concluding 
remarks and future work are presented in Section [6] 

2. THE ANALYSIS MODEL 

As shown in ([T]) and the synthesis sparsity model requires 
that a signal is composed out of only k atoms (columns) of the 
dictionary D, whereas the analysis model requires the signal 
to be orthogonal to a large number I of the rows of the op- 
erator fi. We refer to k as the sparsity of the signal x in the 
dictionary D, and, following ||6l, to Z as the cosparsity of x 
with respect to the operator VI. 

For N < d the analysis and synthesis reconstruction prob- 
lems are shown in |[T1 to be equivalent, with D and Vt being 
pseudo-inverses to each other, D = it^ . However, in general 
for N > d (i.e. is a "tall" matrix) the equivalence no longer 
holds, with (|4]) and (|5]l leading to different solutions. 

The similarity of the two models emerges from the fact 
that both are instances of the Union-of-Subspaces (UoS) 
model |l6l. The set of all /c-sparse signals in a dictionary D 
comprises the union of all the (^) /c-dimensional subspaces 
spanned by any subset of k atoms from the N atoms of D. 
The set of all Z-cosparse signals of an operator VI is the union 
of all the (^) {d — Z) -dimensional subspaces that are the 
orthogonal complements of the subspaces spanned by any I 
rows. We may say, therefore, that the synthesis model is es- 
sentially described by the subspaces where the signal may lie, 
i.e. the non-zero coefficients of the decomposition, whereas 
the analysis model describes the subspaces where the signal 
cannot lie, i.e. the rows that are orthogonal to the signal ^61. 

3. ANALYSIS-BY-SYNTHESIS (ABS) APPROACH 
FOR EXACT RECOVERY 

3.1. Augmented equivalence theorem 

Let us consider the case of reconstruction with exact con- 
straints, i.e. e = in (|5]). The following theorem establishes 
the equivalence between the analysis recovery problem and a 
synthesis recovery problem with a set of extra constraints. 

Theorem 3.1. The solution of the analysis recovery problem 
with exact constraints and full-rank operator VL 



argmin ||r2a;||o with y — Mx 



(6) 



is identical to the solution of the augmented synthesis recov- 
ery problem 



Z? argmin II7II0 withy = Ay 



(7) 



where D ~ Vl\ y 



, A 
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any projector on the nullspace of D. 



with Pd being 



Proof. We show the equivalence of (|6]l with (|7]), starting from 
the approach in 1 1 1. Making the notation Vlx — 7, it follows 
from il^il = Id that x = fl^^. We proceed to substitute 
the unknown variable x in (|6]l introducing 7 instead, but in 
doing that we must keep in mind that 7 is allowed to live only 
in the column span of il, which we can express as the extra 
constraint 7 flfl^j. Therefore we arrive to 

with y = Mn^"f. 



X — il,^ arg min ||7||o 
7:7— ^^ot'-)' 



(8) 



We rewrite the constraint 7 = ilil^7 as = {In — VlQ^)j. 
We can join this with the constraint y ~ Af 51^7 and construct 
a single augmented constraint system 



(9) 
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Let us define D = D,^ . The lower constraint — {In — 
i7J7^)7 is equivalent to 7 living in the column space of il, 
i.e. being orthogonal to the nullspace of D — fl^ (denoted as 
nn); therefore this constraint can be expressed as = Pd7 
with Pd being any projector on ud- Replacing fi^ with D 
and rewriting ([SJ with the augmented constraint (|9| yields 



D argmin ||7||o with 



which is what we wanted to prove. 
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Theorem 3.1 reveals that analysis recovery is a particu- 
lar instance of synthesis recovery; indeed, without the lower 
constraint ( [TO] i would be identical to synthesis-based recov- 
ery. What is specific of the analysis recovery is, therefore, the 
restriction of the solution search space to the column space of 
Vl (or, equivalently, to the row space of D = Vl^). A similar 
condition is used in ifTOl in the context of local optimality of 
analysis operator learning. In practice, this constraint can be 
expressed by finding a set of {N — d) linearly independent 
vectors from (e.g. by finding a SVD decomposition of 
D) and then imposing that 7 is orthogonal to all of the vec- 
tors in this set. Moreover, if D is a tight frame allowing fast 
multiplications via fast transform algorithms, the row vectors 
of Pd can be selected as the "missing" orthogonal rows, thus 
allowing possible fast solver implementations. 

As a consequence of Theorem 1 3.1 [ one can use synthesis- 
based solvers to find the solution for analysis-based recovery. 
While the more general character of synthesis over analysis 
recovery, as well as the subspace restrictions implied by the 
latter, is already known 0] O, to our knowledge this is the 
first time that the equivalence of analysis exact reconstruction 
with an augmented synthesis problem has been stated explic- 
itly and also used as a method for analysis recovery. 

One observes that whenever N < d, ilil^ = In and 
thus the lower subspace constraint in Q vanishes, straight- 
forwardly confirming the equivalence of analysis-based and 
synthesis-based recovery already shown in 1 1 1 for this case. 



Algorithm 1 Proposed Analysis-By-Synthesis (ABS) ap- 
proach for exact reconstruction 

Input: Analysis operator il, measurements vector y, mea- 
surement matrix M 
Output: Recovered signal 

X = argmin |ir2x|lo with y = Mx 

X 

1: Define D = and compute a basis for the null space of 
D using the SVD decomposition, arranging the vectors 
as the rows of a {N — d) x N matrix denoted as Pd 

2: Create augmented constraint matrix A and measurement 
vector y 



3: Solve 

X ~ argmin ||7||o with y — A-y 
1 

using a synthesis-based solver. 



3.2. Proposed approach 

Our proposed approach for analysis recovery with exact con- 
straints (e w 0) is summarized in Algorithm[T[ which we de- 
note as Analysis-By-Synthesis (ABS). It consists of building 
the augmented constraint matrix A and measurement vector y 
and then solving with a synthesis-based algorithm. 

4. EXPERIMENTAL RESULTS 

4.1. Setup 

A significant advantage of our approach is the ability to use 
existing £o or solvers designed for the synthesis reconstruc- 
tion problem, in the third step of Algorithm [T] We run four 
different synthesis-based solvers in the proposed ABS ap- 
proach: Orthogonal Matching Pursuit (OMP) ifTTl with stop- 
ping criterion being a fixed number k of selected atoms (de- 
noted as OMP-Zc), OMP with stopping criterion being residual 
error below 10"^ (OMP-e), Two Stage Thresholding (TST) 
IIT2II (a generalization of CoSaMP and subspace pursuit) and 
Basis Pursuit (BP) [3J for minimization from |,13il . For ref- 
erence we compare with the results obtained with the Greedy 
Analysis Pursuit (GAP) [9| algorithm, which is specifically 
designed for solving the analysis recovery problem dkectly. 

We investigate the phase transition border of the above 
mentioned algorithms for perfect recovery, for the case of ex- 
act reconstruction. The dimension of the signals is fixed to 
200. The analysis operator is created as the transposition of 
a random tight frame, having N = 240 rows. We define the 
parameters 5 = and p — that define the compression 



ratio and the relative cosparsity. For every pair (5, p) we gen- 
erate 100 signals Xi such that ||rixi|lo = N —I and we project 
them using a random measurement matrix M of size m x d, 
with zero-mean unit-norm normal i.i.d. random elements. We 
then attempt reconstruction with the above mentioned algo- 
rithms. For OMP-k we stop after k = N — I atoms have been 
selected. We consider a signal as perfectly recovered if the 
reconstruction error is below 10~^. 

4.2. Resuhs 

Fig|T] displays the percentage of perfectly recovered signals, 
with white indicating 100% recoverability and black 0%. The 
notation ABS indicates that the synthesis solvers are used 
within our proposed approach. 

The results show that our approach is a viable solution 
to analysis-based recovery. However, we find that not all syn- 
thesis solvers are adequate for use with our approach: OMP-A: 
performs poorly, suggesting that this should not be considered 
as an option for recovery. OMP-e, TST and BP provide good 
results, with OMP-e and BP outperforming GAP in some ar- 
eas (lower cosparsity but sufficient measurements, i.e. larger 
6 and larger p), but being outperformed in others (fewer mea- 
surements and higher cosparsity, i.e. smaller 6 and p). 

For completeness, we also present the total running times 
of the algorithms in Table [T] The overall experiment con- 
sisted in recovering a total of 36100 signals ( 19 x 19 pairs 
{S, p) X 100 signals for each) on a 2.83GHz Intel Core 2 Quad 
Q9550 machine running MATLAB 7.7.0. We find that for our 
experiments OMP recovery is the fastest whereas BP is the 
slowest, with TST and GAP yielding intermediate times. 

5. DISCUSSION AND FURTHER CONSIDERATIONS 

As we have seen, the proposed approach is based on rewrit- 
ing analysis recovery as a particular case of the more general 
synthesis recovery problem, subsequently applying a general 
synthesis-based solver Therefore the solver may not fully 
exploit the particularities of the analysis problem, reflected 
in the particular structure of the augmented constraint matrix 
A (i.e. the bottom rows are orthogonal to the upper rows). 
This makes it possible for the results not to be as good as 
with a solver designed exclusively for analysis-based recon- 
struction. For the purpose of this paper, however, we settle 
with the possible shght suboptimality of the synthesis solvers, 
counterbalanced by the increased flexibility conferred by the 
large number of available solvers. 

For reconstruction with approximate constraints, i.e. e > 
in (|5]l, an extra precaution is required when handling the 
augmented constraint matrix in (|9|. We still require that the 
solution 7 satisfies the lower subspace constraint as precisely 
as possible, but we allow a certain degree of approximation 
error for the upper part. This prevents a direct application of 
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Fig. 1. Percentage of perfectly reconstructed signals for analysis-based recovery with different algorithms: our proposed ABS 
approach with four different solvers ( (a), (b), (c) and (d) ) and the GAP algorithm ||9l (e). White indicates 100% recoverability 
and black 0%. 



Table 1. Total running times (x 10^ seconds) 
ABS: 

OMP-fc OMP-e TST BP GAP 
7.691 8.004 22.829 51.152 13.065 



Theorem 3.1 We are currently working on establishing a sim- 



ilar equivalence relation for the case of approximate recovery. 



6. CONCLUSIONS AND FUTURE WORK 

In this paper we have presented a new approach to analysis- 
based exact signal recovery, by reformulating it as a particular 
synthesis-based problem. We prove that, for reconstruction 
with exact constraints, analysis recovery is equivalent to syn- 
thesis recovery with an augmented constraint matrix. This 
means that we can use synthesis-based algorithms for analy- 
sis recovery. Experimental results show that our approach is 
a viable alternative for analysis-based reconstruction. 

For future work, it will be interesting to investigate which 
algorithms are suitable for this approach and the reason why 
some, such as OMP-A;, are performing poorly, while others 
provide good results. We also aim to extend the equivalence 
for approximate recovery. 
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